A State-Compensated Deep Deterministic Policy Gradient Algorithm for UAV Trajectory Tracking

The unmanned aerial vehicle (UAV) trajectory tracking control algorithm based on deep reinforcement learning is generally inefficient for training in an unknown environment, and the convergence unstable. Aiming at this situation, a Markov decision process (MDP) model UAV established, state-compensated deterministic policy gradient (CDDPG) proposed. An additional neural network (C-Net) whose input compensation state output action added to of (DDPG) assist exploration training. It combined DDPG with compensated C-Net as interact enabling rapidly track dynamic targets most accurate continuous smooth way possible. In addition, random noise basis generated behavior realize certain range make value estimation more accurate. OpenAI Gym tool used verify proposed method, simulation results show that: (1) method can significantly improve efficiency by adding effectively accuracy stability; (2) Under same computer configuration, computational cost basically that QAC (Actor-critic behavioral Q) algorithm; (3) During process, accuracy, about 70% higher than DDPG; (4) experiment, under time, error after stabilization 50% lower DDPG.